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Abstract 

The problem of secure multiterminal source coding with side information at the eavesdropper is 
investigated. This scenario consists of a main encoder (referred to as Alice) that wishes to compress a 
single source but simultaneously satisfying the desired requirements on the distortion level at a legitimate 
receiver (referred to as Bob) and the equivocation rate -average uncertainty- at an eavesdropper (referred 
to as Eve). It is further assumed the presence of a (public) rate-limited link between Alice and Bob. In 
this setting, Eve perfectly observes the information bits sent by Alice to Bob and has also access to a 
correlated source which can be used as side information. A second encoder (referred to as Charlie) helps 
Bob in estimating Alice's source by sending a compressed version of its own correlated observation via 
a (private) rate-limited link, which is only observed by Bob. For instance, the problem at hands can be 
seen as the unification between the Berger-Tung and the secure source coding setups. Inner and outer 
bounds on the so called rates-distortion-equivocation region are derived. The inner region turns to be 
tight for two cases: (i) uncoded side information at Bob and (ii) lossless reconstruction of both sources 
at Bob -secure distributed lossless compression. AppUcation examples to secure lossy source coding of 
Gaussian and binary sources in the presence of Gaussian and binary/ternary (resp.) side informations 
are also considered. Optimal coding schemes are characterized for some cases of interest where the 
statistical differences between the side information at the decoders and the presence of a non-zero 
distortion at Bob can be fully exploited to guarantee secrecy. 
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I. Introduction 

Consider the classical problem of compressing a source at a sensor node (referred to as Alice) 
which must be estimated at a remote destination (referred to as Bob) within a certain distortion 
level. Assume also that a (public) rate-limited link is available between the two devices. In 
addition to this, the encoder wishes to leak the least possible amount of information about its 
source to an eavesdropper (referred to as Eve) e.g., an untrusted sensor, who perfectly observes 
the information bits sent by Alice and may have access to an observation correlated to the source. 
Another sensor (referred to as Charlie) will help Bob in estimating Alice's source by sending 
a compressed version of its own correlated observation on a (private) rate-limited link, which 
is only observed by Bob. In this setting, the correlation between the observations can be useful 
not only to decrease the rate needed for the communications, but also to increase secrecy, which 
means the average uncertainty of Eve about Alice's source. From a theoretical viewpoint, the 
problem at hands is therefore very rich and still quite open, as it contains, as subproblems, the 
long-standing information-theoretic problem of distributed lossy source coding, as well as recent 
ones e.g., source coding with security constraints. 

Slepian and Wolf [IJ introduced the problem of distributed lossless compression i.e., when 
Bob wants to perfectly estimate both sources of Alice and Charlie. Wyner [j2J and Ahlswede 
and Komer Q characterized the achievable region when only one source is to be estimated i.e., 
source coding with coded (or partial) side information. Generalization of the Slepian-Wolf setup 
to arbitrary distortion levels on both sources was introduced by Berger Pl, who provided inner 
and outer bounds on the achievable region which do not match in general. When Bob is intended 
to estimate only one source, Berger et al. [5] provided a new inner bound which was further 
proved in [j6j to be equivalent to the one of [|41, and strictly sub-optimal [7J. Several results 
of optimality were proved in case of uncoded side information dSl, lossless reconstruction of 
at least one source [9|, and in some special cases, including Gaussian sources with quadratic 
distortion measure ^T0\. ifTTI . Over the years, these topics have been the focus of intense study and 
some remarkable progress has been made in theoretical and practical aspects, including general 
frameworks for lossless compression with multiple terminals [fT2]| . [[T3]| . lossy source coding with 
uncertain side information at the decoder [|T4l . [fTSl . lossy compression with partially separated 
encoders [[T6l or with many decoders [fT7l - [fT9l . some results of optimality for Gaussian sources 
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in various contexts [|20l . [|2T]| . as well as the design of nested codes for distributed compression 
e.g., using parity-check [22], lattice [l22l - [|24l . or algebraic trellis [25] codes. Nevertheless, in 
spite of these efforts, the simplest scenario of distributed lossy compression first introduced in [4] 
still remains open. 

On the other hand, extensive research has been done on secure communication. The traditional 
focus was on cryptography, based on computational complexity where security only depends on 
the intractability assumption of some hard problems (e.g., factoring large integers). As a matter 
of fact, the security requirements were only taken into account in the upper layers of the OSI 
model (e.g., the application layer), assuming that reliable communication/compression schemes 
were already available. Shannon in [26] introduced the information-theoretic notion of secrecy, 
where security is measured through the equivocation rate -the remaining uncertainty about the 
message- at the eavesdropper. This information-theoretic approach of secrecy allows to consider 
security issues at the physical layer, and ensures unconditionally (regardless of the eavesdropper's 
computing power and time) secure schemes, since it only relies on the statistical properties 
of the system. Adopting this approach in a channel coding perspective, Wyner introduced the 
wiretap channel in [|27l and showed that it is possible to send information at a positive rate 
with perfect secrecy as long as the channel of the eavesdropper is a degraded version of the 
legitimate user's one. Csiszar and Komer [28] extended this result to the setting of general 
broadcast channels with any arbitrary equivocation rate. Since then, several extensions have 
been proposed e.g., for fading channels [|29l , arbitrary i.e., not necessarily stationary memoryless, 
channels [|30l . channels with state information at the encoder [l3TI . cooperative relay broadcast 
channels fl32l (see also [|33l - [l35l for a review of recent results), as well as practical coding 
schemes for secure communication e.g., nested codes for (Gaussian and binary) type-II wiretap 
channels [|36l . LDPC [|37l and lattice ll38l codes for the Gaussian wiretap channel, polar codes 
for binary symmetric channels [39], and construction of secure codes from ordinary channel 
codes [|40l . So far, very few work has been reported on source coding problems with security 
constraints, while early work [HTI . [|42l showed that the presence of correlation between the 
different observations may guarantee some secrecy. 

Researchers have employed two approaches in the literature of secure source coding. In fact, it 
is assumed either that there already exists a secure rate-limited link between Alice and Bob, which 
allows the system to use secret keys, or at least the decoders have access to some side information 
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about the source. In the scenario of secret key sharing, both lossless and lossy compression have 
been studied in various contexts p3 l -p7 l . Classical lossy source coding followed by encryption 
using the secret key was proved to be optimal when the receivers have no side information p&j. 
For the second scenario, recent work [ff8l considered the case of lossless source coding with 
(uncoded) side information at both decoders under the assumption of no rate constraint in the 
communication between Alice and Bob. In such a case, the usual Slepian-Wolf scheme is proved 
to be insufficient. Lossless source coding with coded side information, resp. distributed lossless 
compression, has been studied in [|49l , [|50l . resp. [|5T| . In their "one-sided helper" scenario, 
the authors of [l50l characterized the achievable region when only one source is to be perfectly 
estimated and Eve does not have side information. In particular, they proved that the achievable 
scheme of Wyner [|2j and Ahlswede and Komer [3] achieves the whole region. Inner and outer 
bounds on the achievable region for secure distributed lossless compression have been proposed 
in [|5TI . Secure lossy source coding with side information at the decoders received less attention. 
As a matter of fact, if the (uncoded) side informations at the decoders are degraded then the 
achievable region can be derived as a special case of p7l where Wyner- Ziv coding flS) is optimal. 

In this paper, we investigate the general problem of secure lossy source coding of memoryless 
sources with coded side information at the legitimate receiver in the presence of an eavesdropper, 
who in addition to observe the information bits can also have access to correlated side informa- 
tion, as depicted in Fig.[IJ It is assumed that all links between encoders and decoders are noiseless 
so that they cannot provide any advantage to increase secrecy. This setting can be seen as the 
natural extension of the Berger et al. problem [5J to the one with security constraints. We provide 
inner and outer bounds on the achievable region, referred to as the rates-distortion-equivocation 
region. These bounds do not match in general because of a long Markov chain condition, as 
in JH, From the proposed inner region, we derive two results of optimality for the cases of: (i) 
uncoded side information, generalizing the results in [|48l . Il49l to any arbitrary distortion level, 
and (ii) lossless reconstruction of both sources at the legitimate receiver -distributed lossless 
compression-, refining [ISTl . When dealing with the lossy case in the presence of uncoded side 
information, it should be mentioned here that if one side information (either at Bob or Eve) 
is less noisy than the other, then Wyner-Ziv coding is sufficient. Similarly, for the distributed 
lossless compression setting it is shown that if the side information at Eve is less noisy than the 
observation of Charlie, then Slepian-Wolf coding achieves the whole region. As an application 
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example, we consider the case of secure lossy source coding of a Gaussian source with Gaussian 
side informations, extending [fTOj to the scenario with security constraints. We also consider the 
case of secure lossy source coding of a binary source, where the (uncoded) side information at 
Bob (resp. Eve) is the output of a binary erasure channel (resp. a binary symmetric channel) 
with the source as the input. This model is of theoretical interest since neither Bob nor Eve can 
always be a lessnoisy decoder. 

The rest of this paper is organized as follows. Section |ll] states definitions along with the main 
results on secure lossy source coding with coded side information at the legitimate receiver. 
Section HII] (resp. Section ITVl) provides an optimal characterization of the achievable region for 
the case of uncoded side information at Bob (resp. distributed lossless compression). The detailed 
proofs are relegated to the Appendices as well as a reminder on some useful notions and results. 
Section |V] presents application examples to Gaussian and binary sources. Finally, Section IVTl 
summarizes the paper and provides discussions. 

Notation 

For any sequence {xi)i^n*, notation stands for the collection (x^, x^+i, . . . , x„). x" is 
simply denoted by x". Let T be an arbitrary finite set. The cardinality of T is denoted by ||T||. 
For any subset S C T, notation I5 stands for the indicator function of 5 in T i.e., for each 
t G T, = 1 if t G S, and Is{t) = otherwise. Entropy is denoted by H{-), and mutual 
information by /(■; ■). We denote typical and conditional typical sets by Tg'{X) and T^{Y\x"'), 
respectively (see Appendix lA-AI for details). Let X, Y and Z be three random variables on 
some alphabets with probability distribution p. If p{x\y, z) = p{x\y) for each x, y, z, then X, Y 
and Z form a Markov chain, which is denoted by X -e- Y -e- Z. Random variable Y is said 
to be less noisy than Z w.r.t. X if I{U;Y) > I(U;Z) for each random variable U such that 
U -e- X -e- (Y, Z) form a Markov chain. This relation is denoted by Y Z. For each x G M, 
notation [x]+ stands for max(0;x). Logarithms are taken in base 2 and denoted by log(-). For 
each a, 6 G [0, 1], aicb = a(l — 6) + (1 — a)b. 
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C" Charlie 



K (rate Rc) 



Alice 



J (rate Ra) 



Bob 



i" s.t. E[c/(A",i")] < D 



Eve .iif(A"|JE")> A 



Figure 1: Secure lossy source coding with coded side information. 



II. Secure Lossy Source Coding with Coded Side Information 
A. Definitions 

In this section, we give a more rigorous formulation of the context depicted in Fig. [T] Let 
A, C and S be three finite sets. Alice, Charlie and Eve observe sequences of random variables 
{Ai)i(zfi*, (Cj)jgN* and (Ei)i(=^* respectively, which take values on A, C and S, resp. For each 
i E N*, random variables Ai, Ci and Ei are distributed according to the joint distribution p{a, c, e) 
on A X C X E. Moreover, they are independent across time i. 

Let d : A X A [0; c/max] be a finite distortion measure i.e., such that < d^ax < oo. We 
also denote by d the component-wise mean distortion on x A'^ i.e., for each a", 6" G A^, 

d{a-,h-)^'-Y.Ud{a,M). 
Definition 1: An {n, R a, Rc) -code for source coding in this setup is defined by 

• An encoding function at Alice : {1, . . . , 2"'^^}, 

• An encoding function at Charlie /c : C" -> {1, . . . , 2"-^^}, 

. A decoding function at Bob ^ : {1, ... , 2"^^} x {1, . . . , 2"^^:} ^ j\n_ 
Definition 2: A tuple {Ra, Rc, D, A) G is said to be achievable if, for any e > 0, there 
exists an (n, Ra + £, Rc + e)-code (/a, /c, (7) such that: 

E[d{A\g{fA{A'^),fc{Cn))] <D + e , 
1 



n 



■H{A^\fAiA^),E'')>A-E 



The set of all such achievable tuples is denoted by 71* and is referred to as the rates-distortion- 
equivocation region. 
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w c 



U V A E 
Figure 2: Inner bound-Graphical representation of probability distribution p{uvwace). 

Remark 1: Region TZ* is closed and convex. 

Remark!: Quantities E[rf(A", and i E") in Definition [21 

only depend on the marginal distributions p{a,c) and p{a,e), respectively. The same holds for 
region TZ*. 

B. Inner and Outer Bounds on the Rates-Distortion-Equivocation Region 

The following theorem gives an inner bound on region IZ* i.e., it defines region T^in C TZ* . 

Theorem 1: A tuple (Ra, Rc, D, A) E is achievable if there exist random variables U, 
V, W on some finite sets U, V, W, respectively, s.t. the joint distribution writes p(uvwace) = 
p{u\v)p{v\a)p{w\c)p{ace) , and a function A : V xW ^ A, that verify the following inequalities: 



Ra>I{V-A\W), (1) 

Rc>I{W;C\V), (2) 

Ra + Rc>HVW;AC) , (3) 

D>E[d{A,A{V,W))] , (4) 

A<H{A\VW) + I{A;W\U)-I{A;E\U), (5) 

A-Rc< H{A\V) - I{A; E\U) - I{W- C\V) . (6) 

Region IZ^^ is defined as the convex hull of the set of all such tuples. 



The proof of Theorem [T] is based on superposition coding and random binning at the encoders 
Alice and Charlie, and joint decoding at Bob. The proposed scheme along with standard proper- 
ties of typical sequences enables to characterize the equivocation rate at Eve. The detailed proof 
is relegated to Appendix |Bl The above inner region can also be achieved using a time-sharing 




May 2011 



DRAFT 



SUBMITTED TO THE IEEE TRANSACTIONS ON INFORMATION THEORY 



8 




Figure 3: Achievable tuples {Ra, Rc, ^) for some fixed distortion level D. 

combination of three complementary families of codes. Since this approach may yield better 
intuition, its proof is sketched below. 

Inequalities ([T])-® are identical to the ones of Berger and Tung BU. They ensure perfect 
reconstruction of both variables V and W at Bob, who can hence compute estimate A(y, W) of 
A. The sum-rate constraint (|3]) captures the trade-off between rates Ra and Rq. The information 
must be transmitted by one or the other encoder. 

Let us now give some intuition on Equations ^ and Q. The first term H(A\VW) corresponds 
to the equivocation rate at Bob. Alice thus exploits the admissible distortion at Bob to increase 
the equivocation rate at Eve. Moreover, for given variables V and W, which determine the rates 
and the distortion level at Bob, the auxiliary variable U can be tuned to make Bob more capable 
than Eve i.e., maximize I(A; W\U) — I{A; E\U). This quantity represents the gain (or the loss) 
at Eve in terms of equivocation rate. At the same time. Equation ^ imposes a trade-off between 
the equivocation rate at Eve A and the rate of Charlie Rc, which captures the fact that A cannot 
be too large if Rc is not. If the secrecy requirement is harsh, more information must be sent 
through the private link (between Charlie and Bob). We will refer to quantity A — Rc as the 
public-link secrecy rate. 

Note that Equation © also writes 

A < H{A\UE) - I{V; A\UW) . 
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Table I: Comer points. 



Comer point 


{I) 


{II) 


(III) 


Communication order 


W, U, V 


U, W, V 


U, V, w 


Ra 


IiV;A\W) 


IiU;A) + I(V;A\UW) 


I{V;A) 


Rc 


IiW;C) 


I{W;C\U) 


IiW;C\V) 


D 


E[d{A,A(V,W))] 


E[d{A,A{V,W))] 


E[d{A,A{V,W))] 


A 


H{A\UE) - I{V;A\UW) 


H{A\UE) - I{V;A\UW) 


H{A\UE) - I{V;A\U) 



Variable U is thus considered as a common message i.e., as if Eve could decode it. As a matter 
of fact, in case of uncoded side information at Bob (resp. distributed lossless compression), 
Proposition |3] (resp. lU shows that it is optimal to encode U so that Eve can reliably estimate 
it. The remaining information rate of Alice (on the public link) i.e., I{V]A\UW), is directly 
subtracted from the equivocation rate, meaning that it is treated as "raw" bits of A. 

Sketch of proof of TheoremUl (Time-sharing combination technique): We first construct three 
codes achieving comer points (/), (//) and (///) illustrated in Fig.|3l|4]and|5] Each comer point 
is achieved using a three-step communication scheme which aim is to reliably deliver variables 
(U, V) and W, descriptions of A at Alice and C at Charlie, respectively, to Bob. Note that V is 
on the top of U (superposition coding). At each step, the information previously received (and 
decoded) is used as side-information at Bob. Random binning a la Wyner-Ziv [8] is performed to 
take advantage of this side information. These schemes correspond to all possible combinations 
of the set {U, V, W}, provided that U is decoded prior to V, as summarized in row #2 of Table IH 
For each scheme, the equivocation rate at Eve can be characterized following the argument of 
Appendix IB-H[ After Fourier-Motzkin elimination and classical manipulation, we can prove that 
the three proposed schemes can achieve comer points (/), (//) and (III), which coordinates 
are given in Table IB 

Points (/) and (//) correspond to identical distortion and equivocation rate levels, say D and 
A (see Fig. [5]). By a time-sharing combination of these schemes, each point on segment (/)-(//) 
is also achievable and presents distortion D and equivocation rate A. This segment can be easily 
described since the quantity Ra + Rc is identical for both points (/) and (//) (see Fig. |4]). 

Points (//) and (///) correspond to identical distortion level, say D. By a time-sharing 
combination of these schemes, each point on segment (II)-(III) is also achievable and presents 
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distortion D. This segment can be easily described since quantities Ra + Rc and A — Rc are 
identical for both points (//) and (///) (see Fig. |4] and [5J respectively). 

Segments and (II)-(III) define regions which union is delimited by six hyperplanes 

given by the equations of Theorem [U ■ 

Remark 3: The simple union of the regions given by the equations of Theorem [T] is not convex. 
In fact, a time-sharing variable T cannot be included in auxiliary variables U, V and W. This 
would break the long Markov chain U -e- V -e- A -e- C -e- W which is essential in our coding 
scheme. 

Remark 4: Projections of points (/) and (///) on the plane A = i.e., when there is no 
secrecy constraint, are those obtained using Berger-Tung coding BU. In this case, point (//) is 
useless since it is achievable by a time-sharing combination of points (/) and (///), as shown 
by Fig. m In the general case, the proposed scheme can however improve the security of the 
transmission, as shown in Fig. |5] 

Remark 5: When there is no security requirement, Jana and Blahut [6] recently proved the 
equivalence of the inner bounds of {4\ and (5], meaning that point (/) alone yields the same 
region that points (/) and (///) (after the convex hull operation). A similar result in our secure 
setting does not seem obvious. 

The following proposition gives upper bounds on the cardinalities of alphabets U, V and W. 
The proof, which is given in Appendix O relies on Fenchel-Eggleston-Caratheodory's theorem 
and follow standard cardinality bounding argument (see [l52l Appendix C]). 
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U V A E 




A E 



Figure 6: Outer bound-Graphical representation of probability distributions p{uvace) and 
piwace). 



Proposition 1: In the inner region 71-^ given by Theorem [U it suffices to consider sets U, V 
and W such that ||W|| < ||^|| + 5, ||V|| < {\\A\\ + 5){\\A\\ + 3) and < + 3. 

The following theorem gives an outer bound on region TZ* i.e., it defines region T^out ^ T^*- 
The proof is given in Appendix O 

Theorem 2: Region IZ* is included in T^out^ defined as the closure of the set of all tuples 
{Ra, Rc, D, A) G such that there exist random variables U, V, W on some finite sets 
U, V, W, respectively, and a function A : V x W ^ A satisfying p{wace) = p{w\c)p{ace), 
piuvace) = p{u\v)p{v\a)p{ace) , and 

Ra>I{V;A\W) , 
Rc>I{W;C\V) , 
Ra + Rc> I{VW- AC) , 

D > E[d{A,A{V,W))] , 
A < H{A\VW) + I{A; W\U) - I{A; E\U) , 
A - i?c < H{A\V) - I{A; E\U) - I{W; C\V) . 

As in the classical multiterminal source coding setup flU, the outer region resembles the 
inner region except that it is convex without time- sharing and that Markov chain conditions 
W -e- C -e- {A, E) and U -e- V -e- A -e- (C, E) are weaker than the long Markov chain 
of Theorem [T] (compare Fig. |2] and |6l and see Appendix lA-BI for details on such graphical 
representations). 



May 2011 



DRAFT 



SUBMITTED TO THE IEEE TRANSACTIONS ON INFORMATION THEORY 



12 



C. Special Case: Lossless Reconstruction of A 

In case of lossless reconstruction of A at BobQ if Eve has no side information {E = 0), then 
point (/) yields the optimal performance choosing auxiliary variables U = $ and V = A i.e., 
using Wyner-Ahlswede-Komer coding [|2l, [|3l, as stated by Tandon et al. [50, Theorem 1]: In 
this case, region TZ* writes as the closure of the set of all tuples (Ra, Rc-, D = 0, A) E such 
that there exists a random variable W on some finite set W s.t. W -e~ C -e~ A form a Markov 
chain and 

Ra > H{A\W) , 
Rc>HW;C) , 
A < I{A;W) . 

D. Joint Estimation and Equivocation of Both Sources 

Definition |2] only involves the distortion level at Bob and the equivocation rate at Eve about 
Alice's source. As a matter of fact, the proofs of Theorems H] and |2] can be used to obtain 
inner and outer bounds on the achievable region when also considering a distortion constraint 
on Charlie's source at Bob. This requires the following additional inequality in the definition of 
the achievability: 

^dc{C-,9cifAiA^)JciC-)))] <Dc + e , 

for some distortion measure dc and decoding function gc- The resulting bounds will only differ 
from the ones of Theorems [Hand l2]by adding the following inequality: 

Dc>¥.[dc{C,C{y,W))\ , 

for some function (7 : V x W — C. For the sake of readability, we did not include this fifth 
dimension in the main definitions. In Section El we remove the distortion, addressing the 
case of lossless reconstruction of both sources, and prove that region IZia yields an optimal 
characterization of the corresponding achievable region. 
Furthermore, the joint equivocation rate writes: 

n n n 

'This case is included in the general setup choosing d as the Kronecker delta and D — Q. 
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Alice 



J (rate Ra) 



Bob 



i" s.t. E[c/(A",i")] < D 



Eve >IH{A^\JE^)>1^ 



Figure 7: Secure lossy source coding with uncoded side information. 

and the last term ^H{C'^\A^E^) is constant i.e., independent of the coding scheme. Hence, the 
results involving directly apply to the joint equivocation rate. 

III. Secure Lossy Source Coding with Uncoded Side Information 

A. Definitions 

In this section, we consider the special case depicted in Fig. |7] where Bob has access to uncoded 
side information i.e.. Bob and Charlie are collocated. We need the following new definitions: 
Definition 3: An (n,i?^)-code for source coding in this setup is defined by 
. An encoding function at Alice f : {1, . . . , 2'^^^}, 

. A decoding function at Bob g : {1, . . . , 2"^-4} x C" A"". 

Definition 4: A tuple {Ra, -D, A) G is said to be achievable if, for any e > 0, there exists 
an (n, Ra + £:)-code (/, g) such that: 

E[d(^",(?(/(A"),C"))] <D + e, 

-if(A"|/M"),E") >A-e. 
n 

The set of all such achievable tuples is denoted by T^uncoded referred to as the rate- 

distortion-equivocation region. 

B. Optimal Characterization 

In the setup considered in this section, the following theorem provides a single-letter char- 
acterization of region T^uncoded- The achievability follows from the one of Theorem [1] choosing 
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Figure 8: Projection on the plane Rc = 0. 

auxiliary variable W = C, and removing constraints on Rc (letting Rc tend to oo) i.e., from 
the achievability of point (/) (see Fig. [8]). A new proof is needed for the converse part (see 
Appendix IE]). 

Note that if Eve is a legitimate decoder that wishes to estimate source A within a certain 
distortion criterion (instead of an eavesdropper that other terminals must contend with), then 
[fT9l provides inner and outer bounds on the corresponding rate-distortion function (with two 
decoders and side-information). Finding an optimal characterization of the achievable region in 
such a case is still an open problem. 

Theorem 3: Region T^uncoded writes as the closure of the set of all tuples (Ra, D, A) G M.^ 
such that there exist random variables U, V on some finite sets U, V, respectively, and a function 
A -.V X C ^ A such that U ^ V -e~ A ^ {C, E) form a Markov chain and 

Ra>I{V;A\C), (7) 
D > E[d{A,A{V,C))] , (8) 
A<H{A\VC) + I{A;C\U)-IiA;E\U) . (9) 

Comments similar to the ones of Section III-BI about Theorem [H are also relevant here: 
Equations (|7]) and ([8]) are classical in rate-distortion theory, Alice can exploit the admissible 
distortion at Bob to increase the equivocation rate at Eve (see term H(A\VC) in Equation Q), 
and auxiliary variable U can be tuned to maximize I{A; C\U) — I{A; E\U). 

The following proposition gives upper bounds on the cardinalities of alphabets U and V. The 
proof is similar to the one of Proposition [1] (given in Appendix O and is therefore omitted. 
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Proposition 2: In the single-letter characterization of the rate-distortion-equivocation region 7?.*jjj.ojgj 
given by Theorem |3l it suffices to consider sets U and V such that < \\A\\ + 2 and 

||v||<(M|| + 2)(M|| + i). 

C. Alternative Characterization 

The following proposition can be easily proved from Theorem [31 

Proposition 3: Region 7^*ncoded writes as the closure of the set of all tuples {Ra, D, A) G 
such that there exist random variables U, V on some finite sets U, V, respectively, and a function 
A:V X C ^ A such that U ^ V A ^ {C, E) form a Markov chain and 

Ra > \l{U; C) - I{U- E)] + I{V- A\C) , (10) 
D > E[d{A,A{V,C))] , (11) 
A < H{A\VC) + I{A; C\U) - I{A; E\U) . (12) 

Proof: Inequalities (fT0l)-(fT2l) yield a smaller region than (IT])-®. The achievability of the 
above proposition thus follows from the one of Theorem |3] 
Notice that the r.h.s. of dH) and writes 

H{A\VC) + /(A; C) - I{A; E) - \l{U] C) - I{U; E) 

Maximizing this term w.r.t. U thus boils down to minimizing /(f/; C) — I{U] E). In the worst 
case, setting f/ = makes this term zero, meaning that the optimal choice U* always leads to 
I{U*; C) - I{U*; E) < 0, and makes Equations © and ^ identical. ■ 
Proposition [3l along with the above proof, indicates that the optimal choice of f/ is a random 
variable U* that can be decoded by Eve. Since minimizing quantity I(U ; C) — I{U ; E) w.r.t. U 
corresponds to looking for a part of V which conveys more information about E than C, this 
common message should however give little information to Eve. 

D. Special Cases of Interest 

1 ) Lossless secure source coding: In case of lossless reconstruction of A at Bob, the following 
corollary directly follows from Theorem |3] 
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Corollary 1: In case of lossless reconstruction of A at Bob, region TZ'^^^^if.^ reduces to the 
closure of the set of all tuples {Ra, D = 0, A) E M.^ such that there exists a random variable U 
on some finite set U, such that U -e~ A -e- (C, E) form a Markov chain and 

Ra > H{A\C) , 
A < I{A;C\U)-I{A;E\U) . 

Remark 6: In case of a noiseless public link of unlimited capacity i.e., Ra oo, the authors 
of [|48]| studied the so-called leakage rate, defined as liminf ^I{A^] JE^), which equals H{A) — 
A. Their result "When Bob remains silent" [|48l Theorem 1] thus follows as a special case of 
Corollary [T] 

2) Bob has less noisy side information than Eve (C E): 

Corollary 2: If Bob has less noisy side information than Eve, then region TZl^coded reduces to 
the closure of the set of all tuples (Ra, D, A) E such that there exist a random variable V 
on some finite set V, and a function A : V xC ^ A such that V A-e~ (C, E) form a Markov 
chain and 

Ra>I{V;A\C) , 
D > E[d{A,A{V,C))] , 
A < H{A\VC) + I{A; C) - I{A; E) . 

In this case, random variable U of Theorem |3] is set to a constant value, and hence Wyner-Ziv 
coding lO achieves the whole region. 

3) Eve has less noisy side information than Bob (E C): 

Corollary 3: If Eve has less noisy side information than Bob, then region T^uncoded reduces to 
the closure of the set of all tuples (Ra, E), A) E M.^ such that there exist a random variable V 
on some finite set V, and a function A -.V xC ^ A such that V -e~ A-e~ (C, E) form a Markov 
chain and 

Ra>I{V;A\C) , 
D > E[d{A,A{V,C))] , 
A < H{A\VE) . 
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C" Charlie 



K (rate Rc) 



Bob (A", C") ^ (A", C") 



— > Alice 



J (rate i?^) 



Eve 



Figure 9: Secure distributed lossless compression. 

In this case, random variable U of Theorem |3] is set to V, and hence Wyner-Ziv coding f8l 
achieves the whole region. The equivocation rate at Eve corresponds to the case where Eve can 
reliably decode V. Here, Alice can only exploit the available distortion at Bob to achieve a 
non-zero equivocation rate at Eve. 



IV. Secure Distributed Lossless Compression 

A. Definitions 

In this section, we consider the case where Bob wants to perfectly reconstruct both sources 
A and C, from messages J and K i.e., distributed lossless compression, as depicted in Fig. |9l 
We need the following new definitions: 

Definition 5: An (n, Ra, Rc)-code for distributed compression in this setup is defined by 

. An encoding function at Alice fA-A"—)-{l,..., 2"^^}, 
, An encoding function at Charlie /c : C" ^ {1, . . . , 2"^^}, 
. A decoding function at Bob ^ : {1, . . . , 2""^^} x {1, . . . , 2""^^} A'' x C". 
Definition 6: A tuple {Ra, Rc, A) G M'^ is said to be achievable if, for any e > 0, there 
exists an (n, Ra + e, Rc + £:)-code (/a, fc, q) such that: 

Pr {gUA{A-), fciCn) 7^ {A\ C")} < 5 , 
1 



n 



i/(A"|/A(A"),E")> A-e 



The set of all such achievable tuples is denoted by T^fossiess referred to as the compression- 

equivocation rates region. 
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B. Optimal Characterization 

In the setup considered in this section, the following theorem provides a single-letter char- 
acterization of region T^iossiess- The achievability follows from the one of Points (/) and (//), 
choosing auxiliary variables V = A and W = C (see Section Hl-BI) . A new proof is needed for 
the converse part (see Appendix 10). 

Theorem 4: Region Tll^^^-^^^^ writes as the closure of the set of all tuples {Ra-, Rc-, ^) ^ ^% 
such that there exists a random variable U on some finite set U verifying the Markov chain 
U -e~ A -e~ (C, E), and the following inequalities: 

Ra > H{A\C) , (13) 

Rc > H{C\U) , (14) 

Ra + Rc> H{AC) , (15) 

A < I{A;C\U) - I{A;E\U) . (16) 

Inequalities (fT3])-(fT5]) resemble the ones of Slepian and Wolf [[B Section III]. They ensure 
perfect reconstruction of both variables A and C at Bob. Depending on the distribution of 
(A, C, E), variable U can be tuned to allow non-zero equivocation rate at Eve (see Equation (fT6l)). 
If the side information at Eve E is less noisy than C i.e., E C, then setting U = Ah optimal, 
and hence Slepian- Wolf coding achieves the whole region (with A = 0). 

In case of uncoded side information at Bob, Theorem |4] directly yields Corollary [T] letting Rc 
tend to infinity. 

Remark 7: As a matter of fact, the above result refines recent ones [l35l . IISTTl which only 
provide inner and outer bounds on T^Jossiess- It should be mentioned here that the outer bound 
of |l35l Chapter 8], [l49l . itSTI is incorrect. We use [ISTj as the main reference, but comments below 
also apply to [l35l, Chapter 8] and [1491 as well. In [51], Equation (5) writes A > [H{A\E)-Ra]+, 
meaning that points with A = are not always included in the considered region, while zero 
equivocation rate is achievable by any coding scheme. This inequality can thus not be proved 
in the converse part. In fact. Equation (29) is derived using H{A^\E^ , J) < A, while only the 
reverse inequality holds. 
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C. Alternative Characterization 

As in Section [nl] for lossy source coding with uncoded side information, here we can also 
provide an alternative characterization of region T^fossiess- The achievability follows from the one 
of Theorem |4l A new proof is needed for the converse part (see Appendix [G]). 

Proposition 4: Region T^fossiess writes as the closure of the set of all tuples {Ra, Rc, A) £ 
such that there exists a random variable U on some finite set U s.t. U -e~ A -e- (C, E) form a 
Markov chain and 

Ra > [I{U; C) - I{U- E)] ^ + H{A\C) , (17) 
Rc > H{C\U) , (18) 
Ra + Rc> H{AC) , (19) 
A < I{A;C\U) - I{A;E\U) . (20) 

This new single-letter characterization means that giving U to Eve is also optimal. The 
corresponding additional rate [/(f/;C) — I{U;E)]+ does not lead to a lower equivocation at 
Eve. This should be considered with reference to known results on the wiretap channel [28], 
[|34l . where the so called common message can be chosen so that Eve also decodes it, without 
changing the achievable region. 

V. Application Examples 

A. Gaussian Sources with Coded Side Information 

Consider the source model depicted in Fig. [10] where the source at Alice is standard Gaussian, 
and observations at Charlie and Eve are the outputs of independent additive white Gaussian noise 
(AWGN) channels with input A, gains pc, Pe, and noise powers (1 — p^), (1 — p%), resp., for 
some < Pc, Pe < 1- 

Although Theorem \T\ is stated and proved for finite alphabet sources, we take the liberty to 
use its statement, with the appropriate quadratic distortion measure i.e., the Euclidean distance 
on M (d{a, b) = (a — 6)^, for each a,b E M), as an achievable region also for Gaussian sources 
(using differential entropy h{-), and considering any equivocation rates A G M). In this setup, 
the rates-distortion-equivocation region is denoted by T^oaussian- Notice that the results should be 
generalizable to more general cases of continuous-alphabet sources. 
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PC Nc ^ A/'(0, 1 - pI) 

<k) 'C- 

A ~ A/'(0,1) 




E 



PE Ne ~ A^(0, 1 - pI) 

Figure 10: A model for Gaussian sources 




Proposition[5]below provides an inner bound on T^-Q^^ggj^^ based on the achievability of point (/) 
(see Section HI^Bl) with Gaussian auxiliary variables. This choice is motivated by [fTOl Theorem 1] 
where Oohama proved that it is optimal when only one source is to be estimated within a certain 
distortion level (with no security constraint). 

Proposition 5: In the Gaussian setup considered in this section, a tuple {Ra,Rc,D,A) E 



X 



is achievable if: 



Ra> ^ 



log 



1 - Pc + Pc 



,2 2-2«C 



D 



A<-log (27re(l-p|)) 



— mm 
2 



log 



log 1 



D 

(1 - Pl) 



1 




D I - pI + pl2-^Rc \ 

Fig. [TT] shows a numerical evaluation of the above inner region setting pc = 0.8, pe = 0.6 
and D = 0.1. 
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A 




Figure 11: Achievable tuples in the quadratic Gaussian case {pc = 0.8, pe = 0.6, D = 0.1). 



Proof: Comer point (/) defines a region TZi^i) given by the following inequalities (see 
Table H in Section HTbI) : 

Ra>I{V-A\W), (21) 
Rc>I{W;C), (22) 
D > E[d{A,A{V,W))] , (23) 
A < h{A\VW) + I{A; W\U) - I{A; E\U) . (24) 

For some fixed Rc > and D > 0, auxiliary random variables U, V and W are chosen so 
that bounds on Ra and A given by Proposition [5] yields a point {Ra, Rc, D, A) in region 71(1). 
More precisely, function A is chosen as the minimum mean square error (MMSE) estimator of 
A given V and W, and auxiliary variables V and W are defined as the outputs of independent 
AWGN channels with respective inputs A and C. The gains of these two channels are tuned to 
meet constraints (|22] ) and (|23T ). respectively. Then, since variables A, E and W are Gaussian, 
either W E, or E ^ 4 W . The upper bound (|24] ) is thus maximized setting f/ = 0, or f/ = F. 

ij Variable W-Rate at Charlie: We first define pw £ [0, 1) by = 1 — 2^^^^-, and choose 
random variable W as follows: 

W = pwC + Nw , 
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where Nw ~ A/^(0, 1 — p^) is an independent random noise. With these definitions. 



Pw 



c 



2) Variable V -Distortion at Bob and Rate at Alice: We then define pv € [0, 1) by 



^-iPwfc?-D ifD<l- ipwPC? , 

otherwise. 



(25) 



and choose random variable V as follows: 

W = pvA + Nv , 

where Ny ~ AA(0, 1 — py) is an independent random noise. Note that if large distortion levels 
are allowed, then Alice will not transmit anything (V — $). 
With these definitions, 

E[d{A,A{V,W))] =Yar[A\VW] 

_ {i-p'v)ii-ipwpcr) 

1 - {PvPwPcf 



and 



I{V;A\W) = -log 



— - log 
2 ^ 



log 



Yscr [A\VW]) 

1 - {pwpcf \ 
Yscr [A\VW] J 

i-pI + p12-'^c 

D 



3) Variable U -Equivocation Rate at Eve: The above rates and distortion level can be achieved 
with the following equivocation rate, depending on the choice of U : 
. IfC/ = 0: 



h{A\VW) + I{A- W\U) - I{A; E\U) 
^h{A\E)-I{V;A\W) 
= ^log (27re(l - pD) - ^ 



log 



i-pI + p12-'^o 



D 
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. lfU = V: 

h{A\VW) + I{A; W\U) - I{A; E\U) 
= h{A\E) - I{V;A\E) 



ilo.(2.e(l-p|))-ilog(l^ 
^ log (27re(l - pD) - i log [l + (1 - p|) 



1 

D 



I - pI + p12-^Ro _ 

where the last equality follows from definition (|25T ) after some straightforward derivations. 
This proves Proposition |5] ■ 
If Eve has no side information i.e., pe = 0, then the inner bound given by Proposition |51 and 
corresponding to Oohama coding [[TOl . is optimal. 

Proposition 6: If pE = 0, then region 7?.Q^uj,j,ian reduces to the set of all tuples {Ra, Rc, D, A) G 



X 



X 



verifying the following inequalities: 



Ra>7; 



log 



pI + p12-'^o 



A<ilog(27re)-i 



D 



log 



1-pI + p12-'^o 



D 



J + 



Proof: The achievability follows from Proposition |5] The proof of the converse part follows 
the argument of [TOl. Let {Ra, Rc, D, A) G and e > 0. There exists an (n, Ra-\-£, Rc~^ 

e)-code {fA,fc,9) s.t.: 



E[d{A^,g{fA{A-)Jc{C-)))] <D + e, 

- h{A^\fA{A"), E^) = - > a - £ . 

n n 



(26) 



Denote by J = /^(A") and K = /c;(C") the messages transmitted by Alice and Charlie, 
respectively. 

1) Rate at Alice: The rate at Alice verifies the following sequence of inequalities: 

niRA + e)> H{J) 

> I{J; 

> h{A'^\K) - h{A''\JK) . 

We now study each term of the r.h.s. of the above equation. First, note that from the Gaussian 
distribution of {A,C) and K = /c(C"). there exists random variables A^^ j ~ A/'(0, 1 — p|.). 
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independent of C" (and hence of K) such that Ai = pcCi + N^a, for each i G {1, . . . , n}. The 
conditional entropy power inequality (EPI) [|52l . [|53l thus yields 

= p^2^'^(^"l^) + 27re(l-p^) . (27) 

On the other hand, the rate at Charlie can be lower bounded as follows: 

n{Rc + e)> H{K) 

= I{K; C") 

Equation (1271) thus yields 

2fM^"m > p2^2^('^(^")-"(«^+^)) + 27re(l - p^) 
= p^27re2-2(^^+^) + 27re(l-p^) . 
Term h^A^l-JK) can be easily upper bounded: 

n 



(") ^ 



< ^ilog(27reVar[A,|J,ir]) 

i=l 
(c) " 1 

< 5^-log {27ceE{A-g,{J,K))') 



1=1 

2 \ n ^-^ 

\ i=i 

< -log(27re(D + £)) , 

where 

• step (a) follows from the chain rule for conditional entropy, 

• step (6) from the fact that conditioning reduces the entropy. 
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• Step (c) from the fact Var [Ai \ J, K] is the minimum mean square error (over all possible 
estimators of A^), for each i E {1, . . . , n}, 

• step (d) from the fact that function log(-) is concave, and Jensen inequality, 

• step (e) from the distortion constraint (|26l i. 
Putting everything together, we proved that 



2) Equivocation Rate at Eve: The above argument also provides an upper bound on the 
equivocation rate: 



Remark 8: In case of uncoded side information at Bob i.e., Rc oo, the inner bound 
provided by Proposition |5] is optimal if pc > Pe i-^-, C E. The authors conjecture that it 
also holds if pc < Pe, while the proof seems more tricky. 

B. Binary Source with (Uncoded) BEC/BSC Side Informations 

Consider the source model depicted in Fig. [12] where the source is binary and the side 
information at Bob, resp. Eve, is the output of a binary erasure channel (BEC) with erasure 



n{RA + e)> - h{A''\JK) 

> ^ log {pI 2vre 2-2(^-+^) + 2vre(l - p^) - \ log (2vre(D + e)) 




n(A-£) < h{A'')-H{J) 




This proves Proposition |6l 



E 

^ 




1 — p 



A 




1 -e 



C 




1 — p 



1 -e 



Figure 12: Binary source with BEC/BSC side informations. 
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2p Apjl-p) h2{p) 1 e 

(«) ' ib) ' (c) ' (rf) 

Figure 13: The different regions as a function of e. 

probability e G [0,1/2], resp. a binary symmetric channel (BSC) with crossover probability 
p e [0, 1/2], with input A. 

This model is of interest since neither Bob nor Eve can always be a lessnoisy decoder for all 
values of (p, e). Let /i2 denote the binary entropy function given by h2{x) = — xlog(x) — (1 — 
x) log(l — x). According to the values of the parameters (p, e), it can be shown by means of 
standard manipulations [|54l that the broadcast channel with input A and outputs (C, E) satisfies 
the following properties (see Fig. [131 ): 

(a) < e < 2p: The side information E is a stochastically degraded version of C, 

(b) 2p < e < Ap{l — p): The side information C is less noisy than E i.e., C ^,4 E, 

(c) Ap{l — p) < e < h2{p): The side information C is more capable than E, i.e., I{A] C) > 
I{A-E), 

(d) h2{p) < e <V. Any of the above relations hold between the side informations C and E. 
Corollary |2] thus provides an optimal characterization of the rate-distortion-equivocation region 

^uncoded when e lies in region (a) or {h). Otherwise, only Theorem |3] applies for the general case 
and variable U is neither constant nor equal to V . 

From now on, let the distortion function at Bob d be the Hamming distance and assume for 
simplicity that the source is uniformly distributed, i.e., Pr {A = 0} = Pr {A = 1} = 1/2. We 
know from the cardinality constraints given in Proposition [2] that it suffices to consider sets 
U and V such that ||W|| < 4 and ||V|| < 12. As a matter of fact, according to the following 
proposition, we can restrict our attention to the auxiliary variables (f/, V) obtained as the outputs 
of a degraded binary symmetric broadcast channel with input A, as depicted in Fig. [141 Notice 
that V is identical to the auxiliary variable used by Wyner and Ziv [[H for the rate-distortion 
function of a binary source in the case where there is no eavesdropper. 
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U 1-/3 y l-a A 
< < 



1 




1 



1-/3 l-a 
Figure 14: Binary auxiliary random variables. 



Proposition 7: In the case considered in this section, region 7^*ncoded reduces to the set of all 
tuples {Ra-, D, A) G such that there exist a, /3 G [0, 1/2] satisfying 

>e(l-/i2(«)) , 
D>ea, 

A < e /;,2(«) + (1 - e) h2{a -k (3) - h2{p -k a -k 13) + h2{p) . 

Proof: The achievability part of Proposition |7] is a direct application of Theorem |3l define 
auxiliary random variables U and V as depicted in Fig. [141 and function AonVxC = {0,l}x 
{0,e,l}by 

c if c 7^ e , 
V otherwise . 

Expressions of Proposition |7] follow after some straightforward derivations. 

The converse part needs more arguments. Let (Ra, D, A) be an achievable tuple. From 
Theorem |3l there exist finite sets U, V, random variables U on W, \^ on V and a function 
A:V^A, s.t. U ^ V ^ A ^ {C, E) form a Markov chain and 

RA>eI{V;A) , 
D > eE[d{A, A{V))] , 

A < eH{A\V) + (1 - e)H{A\U) - H{E\U) + h2{p) . 

The proof of the above expressions is straightforward, and hence it is omitted here. We now 
prove that there exist a, /3 G [0, 1/2] satisfying the inequalities of Proposition IT) 
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1) Rate: Random variable A is uniformly distributed on {0; 1}, thus: 

I{y-A) = H{A)-H{A\V) 
= 1-H{A\V) . 

Since < H{A\V) < H{A) = 1, and function h2 is a continuous one-to-one mapping from 

[0, 1/2] to [0, 1], there exists a G [0, 1/2] such that H{A\V) = /12(a), and 

I{V-A) = l-h2{a) . 

2) Distortion at Bob: Since distortion d is the Hamming distance, we can write: 

E[d{A,AiV))]=FT{AiV)^A] , 
and, from Fano's inequality [j53l: 

/i2(Pr{i(V^)^A})+Pr{i(\/)^A}log(P||-l) > H{A\V), 

i.e., 

h (Pr{i(\/)^A}) >h{a) . 
Function /i2 is increasing on [0, 1/2], and a G [0, 1/2]. The last inequality thus implies 

Pr|i(F) ^A^>a. 

3) Equivocation Rate at Eve: Define r.v. V on {0, 1} as the output of a BSC with crossover 
probability a and input A. Since A is uniformly distributed on {0, 1}, A is also the output of a 
BSC with crossover probability a and input V. From Mrs. Gerber's lemma [1551 , we can write, 
for each u ElA: 

H{A\U = u) = h2(ai.h^\H{V\U = u))^ , 

and hence, 

H{A\U) = ^h2 (^ai<h2\H{V\U = u))^p{u) . 

Following the same argument, since E is the output of a BSC with crossover probability p and 
input A, it is also the output of a BSC with crossover probability p^k a and input V, and: 

H{E\U) = J2^^ (^{pi^a)i<h2\H{V\U = u))^p{u) . 
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Now, for each m G W, < H{V\U = u) < H{V) < 1, and there exists /3„ G [0, 1/2] such that 
H{V\U = u) = h2{(3u)- Consequently, 



u 



(1 - t)H{A\U) - H{E\U) = 5^ [(1 - e) h^ia * /3J - hip * « * p{ 

ueu 

< (1 — e) h2{a * /3) — h2{p^a ^ (3) , 

where (3 = (3u* for some u* G W. 

This proves Proposition |7] ■ 
Remark 9: In this binary case with Hamming distance as distortion measure, an achievable 

distortion level D is an upper bound on the average bit error rate (BER) at Bob (while estimating 

A): 

1 " 

E[d{A^,g{f{An,Cn)]=-J2Pr{A^A,] , 

i=l 

where Ai = (^^(/^(A"), C") is the i-th coordinate of the estimate of A" at Bob. At the same 
time, an achievable equivocation rate A provides a lower bound on the BER at Eve, as shown 
by the following sequence of inequalities: 

-/7(A"|JE") < -H{A''\A'') 
n n 

(b) 1 " 



< - y^H{Ai\A, 



n 

i=l 



(c) 1 



-y^H{Wi\Ai) + H{Ai\WiAi 



n . 

1=1 

i'i) 1 

<-j:Hm 

i=l 



< 



1 " 

i=l 



where 

• step (a) holds for any A" G such that A" -e- (J, E^) -e~ A"- form a Markov chain, 

• step (h) follows from the chain rule for conditional entropy and the fact that conditioning 
reduces the entropy, 

• step (c) from Wi = Ai® Ai, for each i E {1, . . . , n}. 
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Figure 15: Equivocation rate at Eve as a function of the distortion level at Bob (p = 0.1, 
e = h2{v) ~ 0.469). 

• step [d) from identity Ai = Wi® Ai and the fact that conditioning reduces the entropy, 

• step (e) from the fact that function h2 is concave, and Jensen inequality. 

Numerical evaluation: Using the inequalities of Proposition Ul we now numerically compute 
some achievable tuples for p = 0.1 and e = /i2(p) ~ 0.469 (see Fig. [15]). In case of lossless 
compression (columns #1 and #2 of Table [III), the auxiliary random variable V is set to A i.e., 
a = 0. Variable U actually enables a non-zero equivocation level. Now assume that the coding 
rate is limited to a maximum of 80% of the required rate for perfect reconstruction of the source 
(column #3). This induces a distortion of 0.015 at Bob and an equivocation rate of 0.133 bits at 

Table II: Some achievable tuples and corresponding parameters for auxiliary random variables 
(p = 0.1, e = h2{p) ^ 0.469). 





Secure source coding 


Slepian-Wolf 


Secure source coding Wyner-Ziv 


Rate R 


0.469 


0.469 


0.375 


0.375 


Distortion D 








0.015 


0.015 


Equivocation Rate A 


0.039 





0.133 


0.126 


a 








0.031 


0.031 
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0.078 





0.050 
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Eve. Even a small increase in the distortion at Bob can be fully exploited by Alice to achieve 
very significant gains (more than third times in this case) in terms of equivocation rate at Eve. 
Moreover, for distortion levels higher than 0.036, Wyner-Ziv coding actually achieves the optimal 
performance, as shown in Fig. \T5\ 

VI. Summary and Discussions 

In this paper, we have addressed the general problem of secure lossy source coding with 
coded side information. Inner and outer bounds on the corresponding achievable region have 
been derived. This setting can be seen as the natural extension of the Berger et al. problem [5] 
by taking the security requirements into account. It should be mentioned here that the latter 
is a fundamental information-theoretic problem for which the best known inner bound is not 
optimal in general. In the same way, our proposed bounds do not match in general, but the 
achievable inner region turns to be optimal for two cases of particular interest. Namely, secure 
lossy source coding with uncoded side information, and secure distributed lossless compression. 
Interestingly enough, it is proved for both cases that there is no loss in coding to provide a 
common description of the source to both receivers, the legitimate one and the eavesdropper. The 
remaining information is intended to the legitimate receiver and considered at the eavesdropper 
as "raw" bits. Furthermore, under certain conditions {e.g., less noisy), the standalone Wyner-Ziv 
(or Slepian-Wolf) coding scheme can achieve the entire region and hence the highest security is 
guaranteed without additional efforts. 

Application examples to secure lossy source coding of Gaussian and binary sources have 
been considered. The binary model is of interest since neither Bob nor Eve can always be a 
lessnoisy decoder and thus the encoding strategy needed to achieve the whole region is rather 
novel. In the Gaussian quadratic case, the results by Oohama [fTOll suggest an inner bound 
which has been proved to be optimal in some cases. A deep analysis along with recent extremal 
inequalities ll56l . [|57l may yield the expected converse. However, in the light of known results 
on Gaussian quadratic multiterminal compression [fTOl . [fTTTl . [|20ll . ETI . [|58ll . this might be a 
tricky problem. 

Several possible extensions of this work can be identified. First of all, we can think about an 
extension of the CEO problem [|59l , [|60l under some security constraints, where the purpose of 
the legitimate decoder is to estimate a common underlying random variable. Recent results [IMI 
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indicate that Wyner-Ziv-like coding works well in this setup, and the quadratic Gaussian case has 
already been solved by Oohama [|62l. Since the quantity of interest is the underlying variable, 
the secrecy of the system could be measured by the equivocation at the eavesdropper about this 
variable rather than the observation of one encoder. 

Further extensions could include the introduction of multiple eavesdroppers in order to consider 
the fact that the encoder cannot reliably know the statistics of the information at the eavesdropper. 
As a matter of fact, if the observations of these multiple eavesdroppers are degraded (or maybe 
less noisy), as it will be the case with scalar Gaussian variables, then a multi-layer superposition 
coding scheme may yield a characterization of the equivocation rate at each eavesdropper. 

Through this work, error- free rate-limited links were assumed between the encoders and 
receivers, while noisy channels could provide additional security, as in the traditional wiretap 
setting. A result of optimality for the case of degraded channels and side informations has 
already been derived 071. A comprehensive study of the more general setup of secure joint 
source/channel coding seems promising. 
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Appendix A 
Useful Notions and Results 

The appendices below provide basic notions on some concepts used in this paper. 

A. Strongly Typical Sequences and Delta-Convention 

Following (631, we use in this paper strongly typical sets and the so-called Delta-Convention. 
Some useful facts are recalled here. Let X and Y be random variables on some finite sets X 
and y, respectively. We denote by Px,y (resp. Py\x, and Px) the joint probability distribution 
of (X, Y) (resp. conditional distribution of Y given X, and marginal distribution of X). 

Definition 7: For any sequence x" G A"" and any symbol a G X, notation A^(a|x") stands for 
the number of occurrences of a in x". 

Definition 8: A sequence x" G A*" is called (strongly) 5-typical w.r.t. X (or simply typical if 
the context is clear) if 



-N{a\x'^) - Pxia] 



< S for each a E X 



n 

and A^(a|x") = for each a E X such that Px{ci) = 0. The set of all such sequences is denoted 
by T,"(X). 

Definition 9: Let x" G X^. A sequence G 3^" is called (strongly) 5-typical (w.r.t. Y) given 
x" if 

1 1 

< 6 for each a E X ,h E y , 



-Ar(a,6|x",i/") - -Nia\x'')PY\x{h\a) 
n n 



and, A^(a, felx"^, y") = for each a E X , h E y such that PY\x{b\a) = 0. The set of all such 
sequences is denoted by TJ'(F|x"). 

Delta-Convention [63]: For any sets X, y, there exists a sequence {5„}„gN* such that lemmas 
below hold@ From now on, typical sequences are understood with 6 = 6n. Typical sets are still 
denoted by T^{-). 

Lemma 1 ( l[63\ Lemma L2.12]): There exists a sequence > such that 



n— >oo 



PximX)) > 1 - r]„ 



^As a matter of fact, Sn — >■ and ^Jn&n — > oo as n — >■ oo. 
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Lemma 2 ( l\63\ Lemma L2.13]): There exists a sequence rin > such that, for each 

n— )-oo 

■ 1 



n 



log\\T^{X)\\-H{X) 



< Vn , 



-log||T,"(F|x")||-/7(r|X) 
n 



< 



Vn 



Lemma 3 (Asymptotic equipartition property): There exists a sequence > such that, 

n— >oo 

for each a;" G T^{X) and each G T^{Y\x''), 



-\ogPx{x'')-H{X) 



n 



n 



logPy|x(y"|x")-if(F|X) 



Lemma 4 (Joint typicality lemma 152]): There exists a sequence ?7„ 



such that 



-logPy(r,"(FK))-/(X;F) 

n 



< rin for each x"" E T^{X) . 



Proof: 



< ||T5"(F|a;")|| 2-"[-^(^)-""l 

(b) 

< 2"[^^(>'l'^) + /3n] 2-"['H'(^)-<^"] 

^ 2-n[I{X;Y)-l3n-an] 



where 



step (a) follows from the fact that T^{Y\x") C Tg'{Y) and Lemma [31 for some sequence 



Or 



• Step (6) from Lemma l2l for some sequence /3„ > 0. 

The reverse inequality Py(T^(Y\x"')) > 2~"['^(^'^)+^"+""l can be proved following similar 
argument. ■ 
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B. Graphical Representation of Probability Distributions 

Following ||64l Section II], we use in this paper a technique based on undirected graphs, 
that provides a sufficient condition for establishing Markov chains from a joint distribution. 
Such a technique for establishing conditional independence was introduced in [1651 for Bayesian 
networks, and further generalized to various types of graphs [|66ll . This paragraph recalls the 
main points of this technique. 

Assume that a sequence of random variables X" has joint distribution with the following 
form: 

= fi{xs^)f2{xs2) ■ ■ ■ fk{xsj , 

where, for each i G {1, . . . , /c}. Si is a subset of {1, . . . , n}, notation xs- stands for collection 
{xj)j^Si, and fi is some nonnegative function. 

1) Drawing the graph: Draw an undirected graph where all involved random variables e.g., 
(Xj)jg{i „}, are nodes. For each i E {I, . . . , k}, draw edges between all the nodes in Xg.. 

2) Checking Markov relations: Let Qi, Q2, and Q3 be three disjoint subsets of {1, . . . ,n}. If 
all paths in the graph from a node in Xg-^ to a node in Xg^ pass through a node in Xg.^, then 
Xg-^ -e- Xg^ -e- Xgg form a Markov chain. The proof of this result can be found in [|64ll and is 
omitted here. 

C. Csiszdr and Korner's Equality 

Lemma 5 (Csiszdr and Korner's equality l{28\ Lemma 7]): Consider two i.i.d. sequences X" 
and F", and a constant C. The following identity holds true: 

n n 

i=i j=i 
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Proof: From the chain rule for conditional mutual information, we can write: 

n n n 

i=l i=l j=i+l 

= J2 I{Yf,X,\CX^-'YJl,) 

i<j 
n j-l 

j=i i=i 

n 

■ 

Appendix B 
Proof of Theorem [H (Inner Bound) 

Let U, V, W be three random variables on finite sets U, V, W, respectively, such that 
p{uvwace) = p{u\v)p{v\a)p{w\c)p{ace), a function A : Vx W — )• A and a tuple {Ra, Rc, D, A) e 
W^. In this section, we describe a scheme that achieves (under some sufficient conditions) tuple 
{Ra, Rc, D, A) i.e., for any £ > 0, we construct an (n, Ra + £, Rc + £)-code (/^, /c, g) such 
that: 

E[d{A^,g{fAiA-)JciC-)))] <D + e , 

-H{A''\fA{A^),E^)>A-e. 
n 

In this scheme, Alice (resp. Charlie) transmits to Bob a compressed version {U,V), with V 
on the top of U, (resp. W) of A (resp. C) using random binning. From the three bin indices. 
Bob jointly decodes variables U, V and W. 

Let e>0, Ri, R2 G R\_ such that Ri+R2 = Ra + e, and 5*1 > S2 > R2, Sc > Rc + 
Define 7 = 

o Umax 

A. Codebook generation at Alice 

Randomly pick 2""^i sequences u"(si) from T^{U) and divide them into 2"^^ equal size bins 
g{i,. -,2"^i}- Then, for each codeword tt"(si), randomly pick 2" ^ sequences S2) 
from T^{y\u^{si)) and divide them into 2"^^ equal size bins {i?2(si, ^2)}^, ^ n 2"^2V 
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B. Codebook generation at Charlie 

Randomly pick 2^^^ sequences w"(s) from T^{W) and divide them into 2"(^c'+e) equal size 
bins {Bc{r)}^^ 

C. Encoding at Alice 

Assume that sequence A" is produced at Alice. Look for the first codeword m"(si) such 
that A") e T^{U,A). Then look for a codeword t;"(si,S2) such that ss), A") e 

T^{V, A\u'^{si)). Let -Bi(ri) and -B2('Si, '''2) be the bins of u"-{si) and f"(si,S2), respectively. 
Alice sends the message J = /^(A") = (ri,r2) on her error-free link. 

D. Encoding at Charlie 

Assume that sequence C" is produced at Charlie. Look for a codeword w"{s) such that 
(w''(s),C") G Tj^(iy,C). Let 5c(r) be the bin of Charlie sends the message K = 

/c(C*") = r on his error-free link. 

E. Decoding at Bob 

Assume that Bob receives J = (ri,r2) from Alice and K = r from Charlie. Look for 
the unique jointly typical codewords w") with bin indices (ri,r2,r) i.e., look for the 

unique indices (si, S2, s) such that f "(si, S2),w"'{s)) G (i?i(ri) x B2{si, x Bc{r)) fl 

Tj^(f/, V, VT). Then compute the estimate g{J,K) E using the component-wise relation 

gi{J, K) = A{vi{si, S2),Wi{s)) for each i = {1, . . . ,n}. 

F. Errors and constraints 

Denoting by E the event "An error occurred during the encoding or decoding steps," we 
expand its probability (averaged over the set of all possible codebooks) as follows: Pr{E} < 
Pq + Pe,i + Pe,2 + Pe,3 + Pd, whcrc cach term corresponds to a particular error event, as detailed 
below. We derive sufficient conditions on the parameters that make each of these probabilities 
small. 

1 ) : From standard properties of typical sequences (see Appendix lA-AI) . there exists a se- 
quence r]n > such that Pq = Pr {(A", C", ^T^{A,C,E)} < r]^. Consequently, 

-Po < 7 for some sufficiently large n. 
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2) : In the first encoding step, Alice needs to find (at least) one codeword such that 

{u^{si), A^) G T^{U,A). The corresponding error probability Pe,i writes: 



Pe,i = Pr{^si s.t. A") G T^{U,A)} 

\ - Pr |([/", A") G T;'([/, A) f/" G T^iU), A" G T5"(A)| 

^ 2-2"-^i Pr{((7",A")eT^"(l/,A) | (7"GT^"(C/),A"eTJ'(A)} 



for some sequence rjn > (see Lemma|4]in Appendix lA-AI) . If Si > I{U ; A), then probability 

n— >oo 

Pe,i vanishes as n tends to infinity, and hence can be upper bounded by 7 for some sufficiently 
large n. 

Similarly, the second encoding step requires condition S2 > I{V;A\U) to succeed with 
probability 1 — Pe,2 > 1 — 7- 

3) : In his encoding step, Charlie needs to find (at least) one codeword w"(s) such that 
(w"(s), C") G Tg'{W, C). Following the above argument, this requires condition Sc > liW; C) 
to succeed with probability 1 — Pg^a > 1 — 7. 

4) : The decoding error probability must be carefully handled. An error occurs when the 
decoded tuple differ from the original one (si, S2, s). There are three meaningful possible events 
so that Pd writes:^ 

Pd = PT{i§^r^^ 

= Pr{{X}U{^,s}U{si,^,s}} 
< Pr {X} + Pr {X, s} + Ft {su0, s} . 
We now study each term of the r.h.s. of the above equation. 

^We denote by s the event "Index s has been correctly decoded", and ^its complement. Same notation holds for indices si, 
S2, and any tuple of indices. 
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Pr{X} = Pr |3 s[,s'2,s' ^ s s.t. 

^ 2n{Si-Ri+S2-R2+Sc-Rc-£) 

X Pr |(f/", 1/", w^") G Tj^(f/, V, VT) (f/", 1/") e r;'(f/, V"), VT" e T/'(vr)| 

2"('^i--Ri+'S'2-'R2+5c-iic-£) 2-"(-f(c^^;^)-'?") 



for some sequence rjn > (see Lemma |4] in Appendix IA-A|) . If Si — Ri + S2 — R2 + Sc — 

Rc — E < I{V; W), then the above probability vanishes as n tends to infinity, and hence can be 
upper bounded by 7 for some sufficiently large n. 



Pt{X,s} = Pr{3 s[ ^ s.t. 

{u^{s[),v-{s[,s',),w^{s)) G (i?i(ri) x B^is'.^r^) x Bc{r)) nTs^{U,V,W)] 

^ 2n(Si-Ri+S2-R2) 

X Pr |([/", V^", 1^") G r5"(f/, V, W) ([/", 1/") G T^{U, V), W G TJ^(l^)} 

Following the above argument, if Si — Ri + S2 — R2 < I{V; W), then the above probability can 
be upper bounded by 7 for some sufficiently large n. 



Pr{si,^,s} = Pr|3 ^ ^ S2 s.t. 

{u^{si),v^{si,s',),w^{s)) G (Biin) X B2isi,r2) x i?c(r)) H TJ^^f/, l^, IV)} 

X Pr |(f/", 1/", W") G T^{U, V, W) V^") G T5'^([/, V^), VT") G TJ^(f/, W)} 

^ 2"('52-'R2) 2-"('f(^;i^|f^)-'?") 

for some sequence r^^ > 0. If S2 — R2 < I{V', W\U), then the above probability vanishes as 

n tends to infinity, and hence Pr{si,^, s} < 7, for some sufficiently large n. 
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5) Summary: In this paragraph, we proved that under some sufficient conditions, Pr {E} < 77. 
G. Distortion at Bob 

We now check that our code achieves the required distortion level at Bob (averaged over the 
set of all possible codebooks): 



E 



d{A-, gifAiA^, fc{Cn))\ < (1 - Pr {E})E [d{A\ s^), ^"(s))) Z] + Pr {E} 

<E[d{AA{v,w))]+l+^4 , 



where the last inequality holds for some sufficiently large n, and follows from Pr {E} < 77, the 
definition of 7, and the argument below: For each (a", f w"') G T^{A, V, W), 
1 



i=l 



1 

n 



d[a, A{v,w))N{a,v,w\a^,v'^,w'') 

(a,v,w)eAxVxW 

= E[d{A,A{V,W))] 

+ d(^a, A{v,w)) ( —N{a,v,w\a^,v'^,w"') — p{a,v,w) 

{a,v,w)eAxVxW ^ 

<E[d{A,A{V,W))]+d^,,\\A\\\\V\\\\W\\Sn , 

where the last inequality holds since (a^,v",w^) E Tg'{A,V,W). The result follows from the 

fact that (A", S2), w^{s)) G T^{A, V, W) when no error occurred, and 5n > (see the 

Delta-Convention in Appendix lA-AI) . For some sufficiently large n, (imax||-4.|| || V|| || >V||5„ < |. 
Condition D > E[d{A, A{V, W))] is thus sufficient to achieve distortion D + e at Bob. 

H. Equivocation rate at Eve 

The equivocation rate at Eve (averaged over the set of all possible codebooks) can be lower 
bounded as follows: 

n n 



1 

n 

(«) 1 
> - 

n 



HiA'^lnE'') - I{A'';r2\riE'' 
'//(A"isiE")-i7(r2) 



> H{A\UE)-R2-e 
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where 

• step (a) follows from the facts that the bin index ri is a deterministic function of the 
codeword index si, the bin index r2 is a deterministic function of A", and conditioning 
reduces the entropy, 

• step (6) for some sufficiently large n, from the fact that the codewords u^{si) are drawn 
i.i.d. (see Lemma |6] below), and r2 G {1, . . . , 2"^^}. 

Condition A < H{A\UE) — R2 is thus sufficient to achieve equivocation rate A — £ at Eve. 
Lemma 6: The following inequality holds for some sequence r]n > 0: 

n— >oo 

> n{H{A\UE) - 7]n) . 

Proof: Since the codeword index si is a deterministic function of A", term H(A'^\siE'^) 
writes 

= - H{si) - H{E''\si) . (28) 

We now study each term of the r.h.s. of the above equation. 
Variables Ai, E^ are i.i.d., hence H{A'^E'') = nH{AE). 

The second term is studied through the distribution of index si, using classical argument of 
typical sequences and random coding. From the encoding procedure described in Section IB-Cl 
the distribution of si writes, for each j G {1, . . . , 2""^^}: 

Pr {s, = j} = Pr I K(j), A^) G T^U, A) n Q (K(^), A^) i T^U, A)) 



where t„ = Pr |(f/", A") G T^{U, A) [/" G T^iU), A" G T^iA)y The entropy of index si 
thus writes 

^(^1) = - - tn)^-Mog(t„(l - t„)'"') - Pe,ll0g(Pe,l) , 

where Pe,i is the error probability of this encoding step. From Section IB-F[ if Si > I{U; A), 
then probability Pe,i vanishes as n tends to infinity. Since each term t„(l— 1„)-'~^ is non-negative 
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and xhgx > , the above entropy can be upper bounded as follows: 

X-I-0+ 

oo 

His,) <-J2 tni^ - tnY-' l0g(tn(l " tn)^"') + v'n^ , (29) 



for some sequence rjn >■ 0. The above series writes 

n— >oo 

oo 

5^t„(l-t„)'-'l0g(in(l-tnr') 

i=i 

oo oo 
= tnlogitn) 5^(1 - tny~' + t„ log(l - Q ^^(j - 1)(1 - tn)^"' • 

i=i i=i 
Equation (|29l ) thus yields the following upper bound: 

< - log(t„) - log(l - tn)^^ + r/i^) . 



n 



Now, from standard results on typical sequences (see Appendix lA-AI) . 2~"(^(^'^)+^" < < 



2-n(i{U;A)-Vn ) foj- go^ie scqucncc r^i^^ > 0. Since iHiOzf) y this yields 

n— >-oo ^ >-0 



f^. c«....... ^(2) . n Since 

< niliU; A) + r/i^)) + 1 + r/f + 



for some sequence r^i"^-* y 0. 

n— ^oo 

The third term can be studied following the argument of [|34] Section 2.3] for the wiretap 
channel. The equivalent of Equation (2.54) of ll34l here writes 



for some sequence r]n^ > 0. 



Equation (1281) along with the above results yields 



-i/(A"|siE") > H(AE) - I(U; A) - r^^^) _ ^ + ^" ^ + ^" ^ _ h(E\U) - 7]^f> . (30) 
n n 



Using the Markov chain U -e- A -e- E, this proves Lemma |6] 
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/. End of Proof 

In this section, we proved that sufficient conditions for the achievability of a tuple {Ra-, Rc-, D, A) 
are given by the following system of inequalities, for each e > 0: 



Ri 
R2 

Ra + £ 



R. 



c 



> 

> 

= Ri + R2 

> 



^1 > Ri 



S2 
Sc 



> R2 

> Rc + £ 



Si — Ri + S2 — R2 + Sc 
Si — Ri - 



c 



51 > I{U-A) 

52 > IiV;A\U) 
Sc > I{W;C) 

e < I{V;W) 



S2-R2 < I{V,W) 

S2~R2 < I{V-W\U) 

D > E[d{A,A{V,W))] 

A < H{A\UE)-R2 

Fourier-Motzkin elimination then yields: 

Ra + s > I{V-A\W) 

Rc + e > I{W-C\V) 

RA + Rc + 2e > I{VW;AC) 

D > E[d{A,A{V,W))] 

A < H{A\VW) + I{A;W\U) - I{A;E\U) 

A-Rc-e < H{A\V) - I{A;E\U) - I{W;C\V) 
This proves Theorem [1] 
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Appendix C 

Proof of Proposition [H (Bounds on the Cardinalities) 
A. Bound on ||W|| 

First, note that the single-letter inequalities of Theorem [Hcan be written as follows: 

Ra>I{V-A\W) , 
Rc > H{C\V) - H{C\VW) , 
Ra + Rc> I{V] A) + H{C\V) - H{C\VW) , 
D > E[d{A,A{V,W))] , 
A < H{A\UE) - I{V; A\W) + I{U; A\W) , 
A-Rc< H{A\V) - I{A; E\U) - H{C\V) + H{C\VW) . 

We then use Fenchel-Eggleston-Caratheodory's theorem and follow standard arguments (see [|52l 
Appendix C]). Consider the following ||C|| + 3 continuous functions of p{c\w): 

p{c\w) , 

I{V;A\W = w) , 

H{C\V, W = w) = H{CV\W = w)- H{V\W = w) , 
E[d{A,A{V,W))\W = w] , 
I{U;A\W = w) 

From Fenchel-Eggleston-Caratheodory's theorem, there exists a random variable W on W with 

||W|| < \\C\\ +3 such that p{c), I{V; A\W), H{C\VW),E[d{A, A{V,W))] Sind I{U; A\W) are 
preserved. 
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B. Bounds on \\U\\ and \\V\\ 

We now rewrite the inequalities of Theorem [Has follows: 

Ra > H{A\W) - H{A\VW) , 
Rc>I{W;C\V) , 
Ra + Rc> C) + H{A\W) - H{A\VW) , 
D > E[d{A,A{V,W))] , 
A < H{A\VW) + I{A; W\U) - I{A; E\U) , 
A - i?c < H{A\V) - I{A; E\U) - I{W] C\V) . 
Consider the following ||^|| + 5 continuous functions of p{v\u): 
p{a\u) = E[p{a\V)\U = u] , 

H{A\VW, U = u) = H{AVW\U = u) - H{yW\U = u) , 
I{W; C\V, U = u)= I{W] C\U = u)- I{W; V\U = u) , 
E[d{A, A{V,W))\U = u] , 
I{A;W\U = u) , 
I{A;E\U = u) , 

H{A\V, U = u) = H{AV\U = u)- H{y\u = u) . 

From Fenchel-Eggleston-Caratheodory's theorem, there exists a random variable U' on VC with 
IIW'II < MII+5 such that j9(a), H{A\VW), I{W; C\V),E[d{A, A{V, W))] , I{A; W\U), I{A; E\U), 
and H(A\V) are preserved. 

Now, for each u' e W, consider the following ||^|| + 3 continuous functions of p{a\u', v): 

p{a\u',v) , 

H{A\W, U' = u',V = v) = H{AW\U' = u\V = v) - H{W\U' = u' ,V = v) , 

I{W;C\U' = u',V = v) , 

E[d{A, A{y,W))\U' = u\V = v] , 

H{A\U' = u',V = v) . 
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From Fenchel-Eggleston-Caratheodory's theorem, there exists a set V with ||V'|| < ||^|| + 3 
and, for each u' eW, a. random variable V |{f/' = u'} on V and a function A'^, : V x W ^ A, 
such that p{a\u'), H{A\VW,U' = u'), I{W;C\V,U' = u'), E[d{A, A{V,W))\U' = u'], and 
H(A\V, U' = u') are preserved. 

Then define set V" = U'x V, random variable V" = (f/', V) and function A" -.V'xW ^A 
by A"{v",w) = A"{u',v',w) = A'^,(v',w). From the above cardinality bounds, ||V"|| < (||^|| + 
5)(||^|| +3). Note that f/'-e-F"-e- A-e- (C, E) form a Markov chain. From these new definitions 
and previous constructions, we check that quantities involving variable V are preserved: 

H{A\V"W) = H{A\U'V'W) 
= H{A\U'VW) 
= H{A\VW) , 

I{W;C\V") = I{W;C\U'V') 
= I{W;C\U'V) 
= IiW;C\V) , 



E[diA,A"{V",W))] 



E[d{A,A'^,{V',W))] 
E\E[d{A,A'u,{V',W))\U'] 
E\E[d{A,A{V,W))\U'] 
E[d{A,A{V,W))] , 



and 



H{A\V") = H{A\U'V') 
= H{A\U'V) 
= H{A\V) . 



This proves Proposition \T\ 
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Appendix D 
Proof of Theorem [2] (Outer Bound) 

In this section, we prove Theorem |2l Let {Ra, Rc, D, A) be an achievable tuple and e > 0. 
There exists an (n, Ra + s, Rc + £:)-code (/yi, /c, g) s.t.: 

m[d{A\g{fA{A-)Jc{C'')))\ <D + e, 

-HiA"\fA{A^),E^)>A~e. 
n 

Denote by J = /^(A") and K = fc{C"') the messages transmitted by Alice and Charlie, 
respectively. From these definitions and the fact that random variables Ai, Ci, Ei are independent 
across time, the joint distribution of (J, A", C", E'^) can be written as follows: 

k, a", c", e") = I/^(a")(j) I/c(c")(A:) d'^, e'-^) p{a„ d, e^) c^+i, e^+J . 

Following the technique described in Appendix lA-BI and using the above expansion, we can 
obtain the graphs of Fig. \T6\ 

For each i G {1, . . . , n}, define random variables Ui, Vi and Wi as follows: 

U, = iJ,A'-\ E'-') , (31) 
V, = {J,A'-\C'-\E'-') , (32) 
W, = [K, C'-' ) . (33) 

From Fig. [161 Ui -e~ Vi -e~ Ai -e~ {Ci, Ei) and Wi -e~ Ci -e~ {Ai, Ei) form Markov chains (see 
Appendix lA-BI for details on this graphical technique for checking Markov relations). 

Following the usual technique, we also define an independent random variable Q uniformly 
distributed over the set {1, . . . , n}, and A = Aq,C = Cq, E = Eq, U = {Q, Uq), V = {Q, Vq), 
and W = {Q, Wq). Note that U ^ V ^ A ^ {C, E) and W C ^ {A, E) still form 
Markov chains, and that {A, C, E) is distributed according to the joint distribution p{a, c, e) i.e., 
the original distribution of {Ai, Ci, Ei). 
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Figure 16: Outer bound-Graphical representation of probability distributions |(a)]p(j, a", c", e"), 
1(b)] a", c", e") and [(c)]p(j, k, aJ\ d\ e"). 



A. Rate at Alice 



(a) 



I{J; KA^'C'E'') 
> /(J; 

n 

= ^I{J] AiCiEi\KA'-^C'-^E'-^) 

i=l 
n 

= ^I{JA'-^E'~^;AiCiEi\KC'-^) - I{A'~^E'-^;AiCiEi\KC'-^) 



i=l 

n 



i=l 
n 



> Y,HV^■,Ai\Wi[ 



i=l 
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where 

• step (a) follows from J = /^(A"), 

• step (6) from the non-negativity of mutual information, 

• step (c) from the chain rule for conditional mutual information, 

. step (rf) from the Markov chain (A^, d, Ei) ^ {K, C'-^) ^ {A'-\ E'-^) (see Fig. [T6bl) . 

• step (e) from the non-negativity of mutual information and definitions (|32|) . (l33l) . 
Using random variable Q, this yields 

1 " 

RA + e>-Y,I{VQ-AQ\WQ,Q = t) 

i=l 

= IiVQ;AQ\WQQ) 
= I{V;A\W) . 

B. Rate at Charlie 

Using similar arguments with K = /c(C'"), we can obtain: 

niRc + e)> H{K) 

= I{K- JA^'C'E'') 
> I{K;A''C''E''\J) 

n 

^^^I{K-AiCiEi\JA'-^C'-^E'-^) 

i=l 

n 

= J2HKC'-^; AiCiEi\JA'-^C'-^E'-'^) 

i=l 
(d) " 

i=l 

where 

• step (a) follows from K = fc{C^), 

• step (6) from the non-negativity of mutual information, 

• step (c) from the chain rule for conditional mutual information, 

• step (d) from the non-negativity of mutual information and definitions (|32l) . (l33l) . 
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Then, using auxiliary random variable Q, 

i=l 

= IiW;C\V) . 

C. Sum-rate 



n{RA + Rc + 2£) > H{JK) 

I(JK- A"C"E"^ 



n 



AidEilA'-'^C'-'^E'-'^) 

i=l 
n 

J2 [l{JKA'-^C'-^E'-^; AidEi) - I{A'-^C'-^E'~^; AiQEi 



id) 



i=l 
n 



J2 HJKA'-^C'^^E'-^; AidEi) 



i=l 
n 



i=l 



where 

. step (a) follows from J = /^(A") and K = fc{C''), 

• step (b) from the chain rule for mutual information, 

• step (c) from the fact that random variables Ai, Ci and Ei are independent across time, 

• step (d) from the non-negativity of mutual information and definitions (|32l) . (l33l) . 
Using random variable Q, this yields 

1 " 

RA + Rc + 2e>-y2 HVqWq; AqCq\Q = t) 
1=1 

= I{VW;AC) . 



D. Distortion at Bob 

Bob reconstructs g{J, K). For each i G {1, . . . , n}, define function Ai as the i-th coordinate 
of this estimate: 

Ai{V,,Wi)^g,{J,K) . 
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The component-wise mean distortion at Bob thus verifies 

D + e> E[d{A'',g{J,K))] 

_ 1 " 
n 



-y2E\d{A,Mv,,w.^) 

%=\ 
1 " 



Q = t 



E d{AQ,AQ{VQ,WQ)) 



E 



d{A,AiV,W)) 



where we defined function A by 

AiV, W) = i(Q, Vq, Wq) ^ Aq{Vq, Wq) . 

E. Equivocation rate at Eve 



n(A -£) < /7(A"iJE") 

= if(A"iJ) -/(A";E"|J) 

J) - /(A"; E") + /(J; E"") 

\JA'-')-I{Af,E,) + I{JE'-';Ei 



(a) 



i=l 
n 



< 



i=l 
n 



1=1 



(d) 



■t=i 



71 

i=l 



JiTA^-^C^-^^^-i) + I{Ai; KC'-^E'-^\JA'-^) - /(A^; E^) + I{JE'-^; Ei 
JKA'-^C'-^E'-^) + I{Ai; KC'-\JA'-^E'-^) 

JKA'-^C'-^E'-^) + /(Ai; KC^-V^'"^^'"^) - Ei\JA'-^E''^] 
Vm) + I{A,- W,\U,) - I{A,- E,\U,[ 



where 



step (a) follows from the Markov chain J -e- A"' -e- i?" (see Fig. I16a|) . 
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• Step (6) from the chain rules for conditional entropy and mutual information, and the fact 
that random variables Ai and Ei are independent across time, 

• step (c) from standard identities and the non-negativity of conditional mutual information, 
. step (d) from the Markov chain Ei -e- Ai -e- ( JA*"^) -e~ E*"^ (see Fig. |16al ). 

• step (e) from definitions (ISTT ). (|32] ) and (|33l) . 
Now, using auxiliary random variable Q, 

2=1 

+ liAg; Wq\Uq, Q = i)- I{Aq; Eq\Uq, Q = i) 
= H{A\VW) + I{A; W\U) - I{A; E\U) . 

F. Public-link secrecy rate 

n(A -Rc- 2e) < H{A''\JE'') - H{K) 
< HiA^lJE"") - H{K\J) 
= H{A''\JE'') - I{K; J) 

n 

=^ J2 [HiAilJA'-^E"") - I{K;AiCi\JA'-^C'-^) 

i=l 

('i) A r • 1 • 111 

- [H{Ai\JA'-^E') - I{K;Ci\JA'~^C'-^) 

i=l 

n 

- Yl [H{Ai\JA'-^C'-^E'-^) - I{Ai;Ei\JA'-^E'-^) 

- I{KC'-^] dlJA'-^C'-^E'-^) 

n 

[h{A,\V,) - E,\Ui) - I{Wf, Ci\Vi)'^ , 

i=l 

where 

• step (a) follows from the fact that conditioning reduces the entropy, 
. step (b) from K = /c(C"), 

• step (c) from the chain rules for conditional entropy and conditional mutual information, 

• step (d) from the non-negativity of conditional mutual information. 
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• Step (e) from the Markov chains Ai -e- {J, A" ^) -e- (C* ^,E^ ^) (see Fig. |16al ) and 

{K, Q) ^ (J, A'-\ C'-i) ^ E'-^ (see Fig. [T6cl) . 
. step (/) from definitions dlB, (EH) and (|33]). 
Using auxiliary random variable Q, 

1 " 

A-Rc-2e<-J2 [h{Aq\Vq, Q = i) - I{Aq- Eq\Uq, Q = i) - I{Wq; Cq\Vq, Q = z) 

i=l 

= H{A\V) - /(A; E\U) - I{W; C\V) . 
G. End of Proof 

We proved that, for each achievable tuple (Ra, Rc, D, A) and each e > 0, there exist random 
variables U, V and W such that U -e~V ^ A^ {C,E) and W ^ C {A, E) form Markov 
chains, and a function A such that 

RA + e>I{V-A\W) , 
Rc + e>I{W-C\V) , 
Ra + Rc + '^£> I{VW- AC) , 

D + e> ¥.[d{A, AiV.W))] , 
A - £ < + /(A; iy|f/) - I{A- E\U) , 

A - i^c- - 2e < H{A\V) - /(A; E|f/) - I{W- C\V) , 

/.e., + £, -Rc + -D + A — £:) G T^out- Recalling that region T^-out is closed, and letting e 
tend to zero prove Theorem |2] ■ 

Appendix E 
Proof of the Converse Part of Theorem [3] 

Let {Ra, D, A) be an achievable tuple and e > 0. There exists an (n, Ra + £)-code (/, g) s.t.: 

E[d{A-,g{f{A-),Cn)]<D + e , 
-i/(A"|/(A"),E") >A-e. 
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Denote by J = f{A") the transmitted message, and define variables Ui and Vi as follows, for 
each z G {1, . . . , n}: 

U, = {J, Cr+i,E*-i), (34) 

Vi = UA^-\C^-\C^^,,E^-') . (35) 

From Fig. I16al Ui -e- Vi -e~ Ai -e- (d, Ei) form a Markov chain. 

We also define an independent random variable Q uniformly distributed over the set {1, . . . , n}, 
and A = Aq,C = Cq,E = Eq, U = (Q, Uq), and V = iQ,VQ). U ^V ^ A^ (C, E) still 
form a Markov chain and {A, C, E) is distributed according to the joint distribution p(a, c, e) 
i.e., the original distribution of {Ai,Ci, Ei). 

A. Rate 

n{RA + e)> H{J) 

= I{J; A"C"E") 
> /(J; 

n 

= /(J; AE,\A'-^C"E'-^) 

i=l 
n 

= J2 [HJA'-'C'-'C^+iE'-';AE,\Ci) - liA'-'C'-'C^^^^.E^-'-AiEilQ) 

i=l 
n 

"^I{JA'-^C'-^Cl'^^E'-^- AiEilQ) 

i=l 
(e) " 

i=l 

where 

• step (a) follows from J = /(A"), 

• step (b) from the non-negativity of mutual information, 

• step (c) from the chain rule for conditional mutual information, 

• step (d) from the fact that random variables Ai, Ci and Ei are independent across time, 

• step (e) from the non-negativity of mutual information and definition (|35]) . 
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Then, using random variable Q, 

1 " 

RA + e>-Y,I{VQ-AQ\CQ,Q^i) 

Th 

i=l 

= IiVQ;AQ\CQQ) 
^mA\C) . 



B. Distortion at Bob 

Bob reconstructs g{J, C"). For each i G {1, . . . , n}, define function as the i-th coordinate 
of this estimate: 

Ai{Vi,Ci) = gi{J,C^ Cj, C]!,_^) . 
The component- wise mean distortion at Bob thus verifies 
L> + £ >E[d(A",^(J,C"))] 

^-y2E\d{A,Mv,,Ci)) 

n L J 

1=1 

1 ^ 

1=1 

^¥.[d{AQ,AQ{VQ,CQ)) 

^W.[d{A,A{V,C))\ , 
where we defined function ^4 on V x C by 

A{v, c) = A{Q, Vq, Cq) ^ Aq{Vq, Cq) . 
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C. Equivocation Rate at Eve 



n{A~6) < H{A''\J,E'') 

= H{A''\J)-I{A'';E''\J) 

= H{A''\JC'') + I {A''] C"| J) - J(v4"; J) 

H{A'yC'') + /(A"; C") - /(J; C") - /(A"; E") + /(J; E'^ 



n 

J2 [H{ApA'-'cn + HA; a) - I{JCr+,; Q) - I{Af, E,) + I{JE^-'; E,) 

i=l 
n 

J2 [HiApA'-'C^'E'-') + liA; a) - /(JClVi; C,) - /(A,; E,) 

i=l 

+ I{JK-'- E,) + I{E,- Cr_^,\JE'-') - I{Cf, E'~'\JCr+i) 

n 

J2 [HiA.lJA'-'C'^E'-') + I{Af, Q) - I{A; Ei) 

i=l 



(J 

i=l 

(J 

i=l 



+ I{E,- JC^^.E^-') - HQ; JC^^.E^ 



n 

id) 



(J 

i=l 



5^ [h{A,\V,Q) + I{Af, Q) - I{Af, Ei) + I{E,- U,) - /(Q; U, 

i=l 
n 

[H{Ai\ViCi) + I{Ai; Ci\Ui) - I{Ai; Ei\Ui[ 



where 

• step (a) follows from the Markov chain J -e- A"' -e- (C", i?"), 

• step (6) from the chain rules for conditional entropy and mutual information, and the fact 
that random variables Ai, Ci and Ei are independent across time, 

. step (c) from the Markov chain {Ai,C') ^ {JA''^) ^ {C'-^,E^^^) (see Fig. and 
Csiszar and Komer's equality [l28l (see Appendix lA-CI) . 

• step {<£) from definitions (l34l) and (|35]) . 

• step (e) from the Markov chain Ui -e~ A^ -e~ (Ci, E^). 
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Using auxiliary random variable Q, this yields 

^-e<-Y,\H{AQ\VQCQ,Q = i) 

1=1 

+ I{Aq- Cq\Uq, Q = i)- I{Aq- Eq\Uq, Q = t) 
= H{A\VC) + I{A; C\U) - I{A; E\U) . 

D. End of Proof 

We proved that, for each achievable tuple [Rai D, A) and each e > 0, there exist random 
variables U, V such that U -e- V -e- A -e- (C, E) forms a Markov chain, and 

RA + e>I{V;A\C) , 

D + 6>E[d{A,A{V,C))] , 

A-e< H{A\VC) + I{A; C\U) - I{A; E\U) . 

Recalling that region 7^*ncoded is closed, and letting e tend to zero prove the converse part of 
Theorem |3l ■ 

Appendix F 
Proof of the Converse Part of Theorem |4] 

Let {Ra, Rc, A) be an achievable tuple and e > 0. There exists an (n, Ra + £, Rc + £)-code 
{fA,fc,g) s.t: 

FT{g{fA{A^)Jc{Cn) ^ < e , 

n 

Denote by J = and K = fc{C^) the messages transmitted by Alice and Charlie, 

respectively. For each i G {l,...,n}, define random variable Ui by 

U, = {J,Cr^„E'~') . (36) 

From Fig. I16al Ui -e~ Ai -e- (Cj, Ei) form a Markov chain. 

We also define an independent random variable Q uniformly distributed over the set {1, . . . , n}, 
and A = Aq,C = Cq,E = Eq, U = (Q, Uq). Note that U ^A^{C,E) still form a Markov 
chain, and that (A, C, E) is distributed according to the joint distribution p(a, c, e) i.e., the original 
distribution of {Ai, Ci, Ei). 



May 2011 



DRAFT 



SUBMITTED TO THE IEEE TRANSACTIONS ON INFORMATION THEORY 58 

A. Rate at Alice 

Following the argument of the converse for the Slepian-Wolf theorem [|53l Section 15.4.2], 
we prove lower bounds on the rates: 

niRA + e)> H{J) 

(a) 

= i7(A"|C") - H{A''\JKC'') 
> nH{A\C) - n0{6) , 

where 

• step (a) follows from the fact that conditioning reduces the entropy, 
. step (b) from J = /a (A"), 

. step (c) from K = 

• step (d) from the fact that random variables Ai and Ci are i.i.d., and Fano's inequalitjc. 

B. Rate at Charlie 

Using similar arguments with K = /c(C'"), we can obtain: 

n{Rc + e)> H{K) 

(a) 

> H{K\J) 

= I{K; C"|J) 

= HiC'lJ) - H{C''\JK) 

(c) " 
i=l 

> J2H{Qm-nO{e) , 

i=l 

where 

• step (a) follows from the fact that conditioning reduces the entropy, 

''Landau-like notation 0(e) stands for a teim X such that < X < fce for some constant k > 0. 
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. Step (6) from K = fciC"), 

• step (c) from the chain rule for conditional entropy and Fano's inequality, 

• step (d) from the fact that conditioning reduces the entropy, and definition (|36l ). 
Now, using auxiliary random variable Q, 

1 " 

^c + e>-Yl H{Cq\Uq, Q = z) - 0{e) 

i=l 

= H{C\U)-0{e) . (37) 

C. Sum-rate 

A lower bound on the sum-rate can be derived as well: 

n{RA + Rc + 2e) > H{JK) 

(b) 

> nH{AC) - nO{e) , 

where 

. step (a) follows from J = /^(A") and K = fc{C''), 

• step (6) from the fact that random variables Ai and Ci are i.i.d., and Fano's inequality. 
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n(A-£) < /7(A"|JE") 

= i7(A"|J) - J(A";E"|J) 

= Hi.A^yjK) + /(A"; K\J) - liA""] E^'IJ) 

< nO{e) + /(A"; C"| J) - /(A"; J) 

= nO{e) + /(A"; C") - /(J; C") - /(A"; E^) + I{J; E") 

nOie) + ^ [/(A,; Q) - /( JCf+i; C,) - I{A; E,) + I{JE'-^- E^) 



i=l 



(d) 



0{e) + [^(^^; Ci) - I{JC^:,^; Ci) - I{A; E,) + I{JE'~'; E^) 

i=l 

+ I{Ei- Cr+,\JE'-') - liQ; E'-Vcr+i) 

n 

0{6) + J2 C,) - liJC^^.E'-'; a) - I{A; Ei) + I{JC^^,E'~^- E,) 



i=l 
n 



nO{e) + ^ \l{A; Q\Ui) - I{A,- E,\Ui) 



i=l 



where 



• step (a) follows from Fano's inequality, and K = fc{C^), 
. step (6) from the Markov chain J ^ A" ^ (C", i?"), 

• step (c) from the chain rule for mutual information, and the fact that random variables Ai, 
Ci, and Ei are independent across time, 

• step (d) from Csiszar and Komer's equality [|28l (see Appendix lA-CK 

• step (e) from definition (|36l) . and the Markov chain Ui -e- Ai -e- (d, Ei). 
Now, using auxiliary random variable Q, 



+ 0(e) 



1=1 



I{A;C\U)-I{A-E\U)+0{e) 
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E. End of Proof 

We proved that, for each achievable tuple {Ra-, Rc, ^) and each e > 0, there exists a random 
variable U such that U -e- A -e~ (C, E) form a Markov chain, and 

Ra + 0{e)>H{A\C) , 
Rc + 0{e)>H{C\U) , 
Ra + Rc + 0{e) > H{AC) , 

A - 0{e) < I{A- C\U) - I{A- E\U) . 

Recalling that region T^fossiess is closed, and letting e tend to zero prove the converse part of 
Theorem |4l ■ 

Appendix G 

Proof of the Converse Part of Proposition [4] 

The proof of the converse part of Proposition |4] follows the same argument that Appendix E 
In particular, definition (|36l) remains the same. The only difference lies in the lower bound for 
the rate at Alice: 

n{RA + e)> H{J) 

= I{J- + /(J; C") 

= iy(A"|C") - H{A'yKC'') + I{J] C") 



(d) 

= —n 



> -nOis) + [HiA,\C.^ + /( JCr+i; Q) 

i=l 
n 



-n 

i=l 

where 



i=l 
n 

"0(£) + Y [h{A,\C{) + I{U,- Ci) - I{Uf, E, 



May 2011 



DRAFT 



SUBMITTED TO THE IEEE TRANSACTIONS ON INFORMATION THEORY 



62 



• Step (a) follows from J = /a^A'^), 
. step (b) from K = /c(C"), 

• step (c) from Fano's inequality, the chain rule for conditional mutual information and the 
fact that random variables Ai, Ci are independent across time, 

• step {<£) from Csiszar and Komer's equality [|28l . 

• step (e) from the fact that random variables Ai, Ci and Ei are independent across time, and 
the non-negativity of mutual information, 

• step (/) from definition (l36l) . 

Using random variable Q and following the argument of Appendix |F1 we proved the following 
lower bound: 

RA + e> H{A\C) + I{U; C) - I{U; E) - 0{e) . 

Since Equation (l37l) still holds, we proved the bound on Ra given by Proposition ID Other steps 
of the proof remain unchanged. ■ 
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